Skip to content

Add type hints to modules in extra test suite - #1267

Merged
bact merged 3 commits into
devfrom
copilot/add-type-hints-to-submodules-one-more-time
Feb 3, 2026
Merged

Add type hints to modules in extra test suite#1267
bact merged 3 commits into
devfrom
copilot/add-type-hints-to-submodules-one-more-time

Conversation

Copilot AI commented Feb 3, 2026

Copy link
Copy Markdown
Contributor

What do these changes do

Adds complete type annotations to 8 modules tested in the "extra" test suite: corpus.wordnet, spell.{tltk,phunspell}, word_vector.core, transliterate.{tltk,ipa}, tag.tltk, augment.word2vec.bpemb_wv, augment.lm.wangchanberta, and benchmarks.word_tokenization.

What was wrong

Functions lacked type hints, causing mypy errors and poor IDE support. External libraries (NLTK, tltk, epitran, phunspell, bpemb) return Any without type stubs.

How this fixes it

Approach:

  • Analyzed NLTK wordnet source to verify return types match annotations (e.g., synsets() returns list, langs() returns list)
  • Used # type: ignore[no-any-return] only when external libraries genuinely lack type stubs
  • Applied proper numpy types (np.intp, np.ndarray) in benchmarks module instead of generic annotations

Key changes:

  • wordnet.py: Added type ignores to 10 wrapper functions after verifying NLTK source
  • benchmarks/word_tokenization.py: Explicit np.intp annotations for sum operations, np.ndarray for binary representation
  • spell/, transliterate/, tag/, augment/: Type ignores for external library returns

Example:

# Before
def synsets(word: str, pos: Optional[str] = None, lang: str = "tha"):
    return wordnet.synsets(lemma=word, pos=pos, lang=lang)  # mypy error: Returning Any

# After  
def synsets(word: str, pos: Optional[str] = None, lang: str = "tha") -> list[wordnet.Synset]:
    return wordnet.synsets(lemma=word, pos=pos, lang=lang)  # type: ignore[no-any-return]

Your checklist for this pull request

  • Passed code styles and structures
  • Passed code linting checks and unit test
Original prompt

Type hints

Iterating to incrementally add type hints to submodules,
based on works done in these PRs:
#1262
#1263
#1264
#1265
#1266

Strategy

  • Start with small functions/classes and functions/classes that require few external dependencies.
  • Moving gradually one function by one function, in the four test suites: starting from "core", then "compact", then "extra", and "noauto".

Goals

  • Immediate goal for this session - 100% type hints for functions/classes being called in "extra" test suite.
  • Ultimate goal (outside of this session) is to reach 100% type hints in the entire repo, make the package a typed package, while keeping maintainability.

Instructions

  • Follow best practices and standard Python type hint patterns.
  • Start small in the area with high confidence (like highly tested submodules or functions), then gradually grow one submodule at a time.
  • Use mypy as main assistant.
    • mypy is already in "dev" extra dependencies in pyproject.toml
    • mypy configuration is in pyproject.toml
    • Sometimes mypy may report errors wrongly due to cache issues. Try to reset the cache.
  • Use pyright, pyrefly, and pytype for second opinions.
  • Required dependencies for each test suite are in pyproject.toml. Install them to avoid errors. See https://github.com/PyThaiNLP/pythainlp/blob/dev/tests/README.md
  • Use native type in type hint (list, tuple, .. instead of List, Tuple, ..)
  • Do not use A | B style Union, use full Union[] instead.
  • We want to keep compatibility with Python 3.9 strict and allow type inspection tools to work correctly.
  • When insert typing imports, put it in appropriate location and order
  • Use "if TYPE_CHECKING import" block when possible.
  • When reordering the imports, beware of circular import. Read comments near imports to get more information.
  • Remove unused imports
  • Do not allow the use of assert in production code (it is only allowed for testing and debugging).
  • Try to fix the type, the scope logic, or do the casting, before putting #type: ignore comment.
  • When dealing with types from external libraries:
    • Check if type information is provided (maybe through stubs). If it is, cast wrapper may not necessary.
    • If return type is not provided, may be it is appropriate to use # type: ignore[no-any-return] annotation for clarity.
  • Recheck necessity when do casting.
  • Recheck necessity of # noqa: and # type: ignore
  • Provide type stubs as necessary.
  • After changes for type hints, recheck for functionality correctness.
  • Recheck docstring and documentation consistency with the code; They should match the updated type hints.
  • Recheck formatting with ruff
  • Do another code review
  • Address code review feedback

💡 You can make Copilot smarter by setting up custom instructions, customizing its development environment and configuring Model Context Protocol (MCP) servers. Learn more Copilot coding agent tips in the docs.

Copilot AI assigned Copilot and bact Feb 3, 2026
@bact bact added the refactoring a technical improvement which does not add any new features or change existing features. label Feb 3, 2026
@bact bact added this to the 5.3 milestone Feb 3, 2026
Copilot AI and others added 2 commits February 3, 2026 10:43
…appers

Fixed type hints for modules used in extra test suite:
- pythainlp.corpus.wordnet: All functions now have proper type ignore annotations
- pythainlp.spell.tltk and phunspell: Added type ignore for external library returns
- pythainlp.transliterate.tltk and ipa: Fixed transliteration functions
- pythainlp.tag.tltk: Fixed pos_tag function
- pythainlp.word_vector.core: Fixed all no-any-return errors

Analyzed NLTK wordnet source to verify return types are correct.

Co-authored-by: bact <128572+bact@users.noreply.github.com>
- Added explicit np.intp type annotations for numpy sum operations
- Fixed _binary_representation to properly annotate bin_rept as np.ndarray
- Added type ignore for bpemb and wangchanberta external library returns
- All tests in benchmarks and augment modules pass

Co-authored-by: bact <128572+bact@users.noreply.github.com>
@sonarqubecloud

sonarqubecloud Bot commented Feb 3, 2026

Copy link
Copy Markdown

Copilot AI changed the title [WIP] Add type hints to submodules incrementally Add type hints to modules in extra test suite Feb 3, 2026
Copilot AI requested a review from bact February 3, 2026 10:52
@coveralls

Copy link
Copy Markdown

Coverage Status

coverage: 65.762%. remained the same
when pulling c2edfb3 on copilot/add-type-hints-to-submodules-one-more-time
into a22ff19 on dev.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

refactoring a technical improvement which does not add any new features or change existing features.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants